[3.0-dev] Backport prepared-statement / connector fixes (Fixes #24186)#24191
[3.0-dev] Backport prepared-statement / connector fixes (Fixes #24186)#24191jiangxinmeng1 wants to merge 13 commits intomatrixorigin:3.0-devfrom
Conversation
…23413) - Return BOOL as TINYINT(1) (0/1) instead of VARCHAR ("true/false") in MySQL protocol - Adjust column definition (type/length/charset) and result row encoding (text/binary) - Update related unit test and add jdbc-type-compatibility case - Refresh BVT expected results Approved by: @XuPeng-SH, @heni02
Adds the missing server_id system variable to MatrixOne so ODBC clients can read @@server_id during connection initialization without error. Also adds regression coverage in the distributed system variable tests. This prevents Tableau/ODBC connections from failing with “system variable does not exist.” Approved by: @fengttt, @heni02, @XuPeng-SH
…stmt null json panic (matrixorigin#23864) When a prepared statement inserts NULL into a JSON column multiple times, the second execution causes a panic during SELECT: ``` panic runtime error: index out of range [0] with length 0 ByteJson.Unmarshal (bytejson.go:96) DecodeJson (encoding.go:93) ColumnSlices.GetStringBased (output.go:538) ``` **Reproduce:** ```sql CREATE TABLE test_ps (id INT, meta JSON); PREPARE stmt FROM 'INSERT INTO test_ps (id, meta) VALUES (?, ?)'; SET @A=1, @b=NULL; EXECUTE stmt USING @A, @b; SET @A=2, @b=NULL; EXECUTE stmt USING @A, @b; DEALLOCATE PREPARE stmt; SELECT meta FROM test_ps where id = 1; -- ok SELECT meta FROM test_ps where id = 2; -- panic ``` **Root cause:** `FunctionExpressionExecutor.doFold()` calls `ToConst()` on the result vector after constant folding. On the second execution, `PreExtendAndReset()` calls `ResetWithSameType()` which clears nsp (null bitmap) but preserves the CONSTANT class. When `FunctionResult.AppendBytes(nil, true)` is called on a CONSTANT vector, it skips setting the null marker entirely, resulting in a non-null empty byte slice being stored instead of NULL. **Fix:** Reset vector class to FLAT in `ResetWithSameType()`, so the vector is in a clean state for re-population. `doFold()` will call `ToConst()` again after evaluation if needed. Approved by: @heni02, @aunjgr
This PR fixes a missing `MYSQL_TYPE_JSON` branch in the prepared-statement binary execute parser on `main`. Without this change, `COM_STMT_EXECUTE` falls through to `unsupport parameter type` when a client sends a JSON-typed parameter through the binary protocol. Focused validation: - `go test ./pkg/frontend -run 'TestParseExecuteDataWithJSONParam'` - `go test ./pkg/frontend -run 'TestParseExecuteDataWithJSONParam|Test_StripPassword'` Approved by: @LeftHandCold
This PR reclaims prepared-statement binary parameter state after `COM_STMT_EXECUTE` and on malformed execute cleanup. It fixes two closely related problems on `main`: 1. repeated binary executes with varlen parameters reused the same params vector and kept appending to its area, growing session memory until statement close; 2. malformed `COM_STMT_EXECUTE` packets could leave a partially populated params vector attached to the prepared statement because the parse-error path dropped the statement pointer before cleanup. Focused validation: - `go test ./pkg/frontend -run 'TestPrepareStmtClearBinaryParamStateReleasesParamArea|Test_ExecRequestStmtExecuteErrorClearsPreparedParamState'` - `go test ./pkg/frontend -run 'TestPrepareStmtClearBinaryParamStateReleasesParamArea|Test_ExecRequestStmtExecuteErrorClearsPreparedParamState|Test_parseStmtSendLongData'` Approved by: @LeftHandCold
This PR fixes three categories of issues: - Repeated COM_STMT_SEND_LONG_DATA chunks for the same parameter now **append** instead of overwriting, matching MySQL protocol semantics. - Truncated COM_STMT_EXECUTE packets now return a malformed-packet error instead of risking a panic while reading the new parameter bound flag. - PreInsert.refreshAutoIncrementTableID() wrote directly to the shared *plan.TableDef.TblId during Prepare(). - MultiUpdate.Prepare() read the same TblId concurrently in a different ants worker pool goroutine. - **Fix**: Store the refreshed TblId in a local container field (ctr.tblId) instead of mutating the shared plan object. - PartitionInsert and PartitionDelete temporarily mutated the shared ObjRef.ObjName during partition iteration (save/restore via defer) -- replaced with stack-local copies. - PartitionMultiUpdate.writeTable() directly mutated shared MultiUpdateCtx entries -- now uses clone() to match the writeS3 code path. - Also fixed a pre-existing copy-paste bug in MultiUpdateCtx.clone() where PartitionCols was incorrectly copying DeleteCols. Approved by: @ouyuanning
This PR hardens prepared-statement binary protocol cleanup on `main`. It fixes three closely related problems in the frontend prepare path: 1. `PrepareStmt.Close()` now keeps a valid process context for freeing binary parameter vectors after binary parsing. 2. `COM_STMT_RESET` / `reset prepare` now clears accumulated binary prepared-statement state instead of acting like a no-op. 3. `COM_STMT_CLOSE`, `COM_STMT_RESET`, and malformed `COM_STMT_EXECUTE` error paths now cleanly return and clean up instead of dereferencing nil state or leaving dirty binary state behind. Focused validation: - `go test ./pkg/frontend -run 'Test_doResetClearsPreparedBinaryState|Test_ExecRequestPrepareCommandMissingStmt|Test_ExecRequestStmtExecuteErrorClearsPreparedBinaryState|TestPrepareStmtCloseAfterBinaryParamParsing|TestMysqlProtocolImpl_Close|Test_parseStmtSendLongData'` Approved by: @LeftHandCold
|
|
Merge Queue Status
This pull request spent 30 minutes 17 seconds in the queue, with no time running CI. Waiting for:
All conditions
ReasonThe merge conditions cannot be satisfied due to failing checks HintYou may have to fix your CI before adding the pull request to the queue again. |
Merge Queue Status
This pull request spent 31 minutes 48 seconds in the queue, with no time running CI. Waiting for:
All conditions
ReasonThe merge conditions cannot be satisfied due to failing checks HintYou may have to fix your CI before adding the pull request to the queue again. |
Merge Queue Status
This pull request spent 20 minutes 55 seconds in the queue, with no time running CI. Waiting for:
All conditions
ReasonThe merge conditions cannot be satisfied due to failing checks HintYou may have to fix your CI before adding the pull request to the queue again. |
What type of PR is this?
Which issue(s) this PR fixes:
fixes #24186
What this PR does / why we need it:
Backports the prepared-statement / connector compatibility fixes from
mainto3.0-devlisted in #24186:server_idsystem variable for ODBCResetWithSameTypeto fix prepared stmt null json panicAdditional changes required to make the cherry-picks compile and pass tests on 3.0-dev:
clearBinaryParamStateruns;doResetnow clears binary state; COM_STMT_RESET / COM_STMT_CLOSE error paths return early instead of dereferencing a nil prepareStmt;parseStmtExecutereturnspreStmton error).MultiUpdateCtx.clone()helper (withPartitionColsinitialization) on 3.0-dev that fix: harden prepare long data parsing #23980 expects.deletion_partition_test.go,insert_partition_test.go) to use the 2-argumenttestutil.MakeInt64Vector/MakeRowIdVectorsignatures present on 3.0-dev, and renamed the partitionExprWithRowIDfield reference toExpr.TestResetWithSameTypeResetsClass/TestFunctionResultAppendNullAfterToConst(already present on 3.0-dev).MYSQL_TYPE_DATETIME/MYSQL_TYPE_TIMESTAMPinappendResultSetBinaryRow2and brought scale/timezone-aware DATE/DATETIME/TIME/TIMESTAMP formatting intoappendResultSetTextRow(matchesmain).Special notes for your reviewer:
CI on this branch will validate the BVT delta. Tested locally with
go test -tags matrixone_test ./pkg/frontend/... ./pkg/sql/colexec/{insert,deletion,preinsert,multi_update}/... ./pkg/container/vector/...— all green (withTZ=UTCfor the pre-existing TZ-sensitiveTestRowToString).